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Turing Machines and Finite Automata^ 

TOMOYUKI YAMAKAMU 
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computational power when randomized advice is provided in place of deterministic advice. 
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O ■ 1 Advice and Weak Computational Models 

When a machine has a clear, limited operational capability, how can we enhance its computational power 
beyond its plausible limitation? A straightforward way is to provide a piece of supplemental external infor- 
mation besides original input data so that such extra knowledge helps the machine solve a target problem 
efficiently. A notion of so-called advice is such additional information, which depends only on the size of 
inputs, given to the underlying machine. Since Karp and Lipton [5] initiated it in early 1980s, the study of 
advice has attracted numerous researchers in the fields of, e.g., computational complexity and cryptography. 
H— I , To grip a better understanding of the roles of the advice, we intend to take a rather simple but direct approach 

I^H ' toward an investigation of the strengths and limitations of the advice, particularly on weak models of advised 

computations. 

One-tape (two-way one-head) Turing machines (or lTMs, in short) running in linear time could be one of 
the most basic types of computational models ever discussed in computational complexity theory. A theory 
of linear-time lTMs has been studied intermittently since mid 1960s (see (10) for references). An immediate 
advantage of studying such weak models is that we can prove anticipated class separations without relying 
on any unproven assumption, such as the existence of one-way functions. Moreover, we can conduct a precise 
analysis of advice when its underlying computation is limited in power. As were shown in [H [SJ IIP) , certain 
variants of this 1TM model are closely tied to one-way finite (state) automata with constant memory space, 
which could be viewed to run a simple form of memoryless online algorithms. Advised computations of one-way 
deterministic finite automata (or ldfa's, in short) were initially studied in [TU] and deterministic linear-time 
lTMs with advice were discussed in [TO] . Interestingly, it was shown in [10] that deterministic linear-time 
lTMs that take linear-size advice are no more powerful than ldfa's with advice of size equal to input size. 
This characterization makes it easier for us to handle the models of linear-time lTMs. Recently, a series of 
studies |12[ 113) revealed the power and limitation of advice, when given to its underlying finite automata. 
(Another direction with advice was recently indicated in pQ.) In addition to standard (deterministic) advice, 
we also study in this paper randomized advice (in which each advice string is chosen at random according to 
a certain probability distribution), which may allow its underlying machines to err with, e.g., bounded-error 
probability (i.e., at most a certain constant probability away from 1/2). A piece of such randomized advice 
gives a significantly high power to the underlying machines. 

Concerning the aforementioned models of linear-time lTMs, we shall focus our study only on the following 
four language families: 1-DLIN (deterministic), 1-BPLIN (bounded-error probabilistic), 1-PLIN (unbounded- 
error probabilistic), and 1-C = LIN (error probability exactly 1/2), introduced in [10] . These language families 
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Figure 1: A hierarchy of advised language families 



can be viewed as "scaled-down" versions of the well-known complexity classes, P, BPP, PP, and C = P. Some of 
their advised counterparts are succinctly denoted as 1-DLIN/Zm, 1-PLIN/Zm, and 1-C=LIN/Zm. Moreover, for 
given randomized advice, we write some of their corresponding families as 1-BPLIN/ i?Zm, 1-PLIN/ i?Zm, and 
1-C =L1N / Rlin. Similarly, based on the finite automata models, we define REG/n and CFL/n respectively as 
the families of regular languages with advice and of context-free languages with advice. We further introduce 
two additional language families equipped with randomized advice: KEG/Rn and CFL/ Rn. 

In this paper, we shall present new collapses and separations among the above-mentioned advised language 
families. Our results are summarized in Figurc[TJ To obtain these results, we shall show new characterizations 
of advised families and also their new structural properties, which are interesting on their own right. We hope 
that this paper opens a door to a rich research area that sits between computational complexity theory and 
formal language and automata theory. 

2 Basic Notions and Notations 

We briefly explain fundamental notions and notations used in the subsequent sections. Let N be the set of all 
nonnegative integers and set N + = N — {0}. For any pair to, n G N with to < n, the notation [to, n]z denotes 
the integer interval {to, rn + 1, . . . ,n}. Conventionally, we write [to] for [1, m]z- Let K-° be the set of all 
nonnegative real numbers. A function / : N — > H.— ° is said to be negligible if f(n) < l/p(n) for any non-zero 
polynomial p and for all but finitely-many numbers n in N. An alphabet is a nonempty finite set £ and a 
string over £ is a finite series of symbols taken from E. Let A express the empty string. Given a string x over 
£ and a symbol a G £, the notation # CT (a;) denotes the number of all occurrences of a in x. The notation x R 
expresses the string x in reverse. The set of all strings over £ is denoted £* and a language over E is a subset 
of E*. For notational convenience, for any language S, we define S(x) = 1 if x € S; S(x) — if x £ S. The 
notation S for the language S is the complement of S; namely, S = E* \ S. The length of a string x, denoted 
\x\, is the total number of occurrences of symbols in x. A length function is a map from N to N. For any length 
n G N, let £" = {x G E* | \x\ = n}. A probability ensemble p over E* is an infinite series {/i ra }ngN, hi which 
each p n is a probability distribution over £™ (i.e., < p n (x) < 1 for any x G E™ with X^es™ l^n( x ) = !)• 

Our basic model of computation is one-tape (or single-tape) two-way one-head off-line Turing machines 
(or lTMs), each of which can be expressed as a sextuple (Q, E, S, q$, Q a cc, Qrej), where Q is a finite set of 
inner states, E is an alphabet, qo (G Q) is the initial state, S is a transition function, Q acc (C Q) is a set of 
accepting states, and Q re j (C Q) is a set of rejecting states. Write Q non for the set Q \ (Q aC c U Qrej)- This 
machine M is equipped with one input/work tape, on which an input string is initially written, surrounded 
by two endmarkers fy and $, and a tape head either moves in both directions (to the left or to the right) or 
stays still, after starting from the left endmarker fy. For our convenience, let E = E U {£, $}. When S is of the 
form 5 : Q non X E — y Q x E x {L, N, R}, M is called deterministic. To the contrary, M is probabilistic if 5 
satisfies 5 : Q non x E — > V(Q x E x {L,N,R}), where V(A) is the power set of A. In this case, each move 
(q, a) M> (q'a' , d) G S(q, a) of M is associated with a transition probability. If M is deterministic (probabilistic, 
resp.), then we succinctly call it 1DTM (1PTM, resp.). The extended transition function S induced from S is 
recursively defined as S(q, A) = q and S(q, xa) — S(S(q, x), a) for each symbol a G E and each string x G E*. 



By adopting Michel's strong definition for machine's running time (S], we say that M runs in linear time if 
the longest computation path (even in a case of probabilistic computations) of M on any input x of length n is 
bounded from above by a certain fixed linearly-bounded function in n; in other words, every computation tree 
of input size n has hight of at most 0(n). (See [10] for more discussions on the topics of this strong definition 
of running time.) 

One-way finite (state) automata are a special case of those linear-time lTMs with the following restrictions: 
(i) a tape head always moves from left to right without stopping, (ii) it halts just after scanning the right 
endmarker $, and (iii) the tape is read-only. Let REG, CFL, and DCFL denote respectively the families of 
regular languages, of context-free languages, and of deterministic context-free languages. 

In this paper, all lPTMs use only rational transition probabilities. We say that a 1PTM M recognizes a 
language L if, for every input string x, (i) if X £ L, then M accepts x with probability > 1/2 and (ii) if x $ L, 
then M rejects x with probability > 1/2, where "probability" is taken over all inner coin tosses of M on the 
input x. Moreover, M has bounded error if there exists a constant (also called an error bound) e £ [0, 1/2) 
such that, for every input string x, either M accepts x with probability at least 1 — e or M rejects x with 
probability at least 1 — e; otherwise, M is said to have unbounded error. Following [10], we denote by 1-DLIN 
(1-BPLIN, 1-PLIN, resp.) the family of all languages that are recognized by lDTMs (lPTMs with bounded 
error, lPTMs with unbounded error, resp.) in linear time. The family 1-C = LIN (pronounced "one C equal 
LIN" ) [TU] is the collection of all languages L for which there exist a linear-time 1PTM M that satisfies the 
following condition: for every input x, X £ L iff M accepts x with probability exactly 1/2. 

To feed a piece of supplemental information together with an input string to lTMs, we use a "track" 
notation of [10] . For two symbols a £ S and t £ T, the notation [ ° ] expresses a new symbol made from 
a and r. For a 1TM equipped with an input/work tape, this symbol [ " ] is written in a single cell, which 
consists of two tracks, whose upper track contains a and the lower track contains r. For two strings x and 
y of the same length n, [ * ] succinctly denotes the string [ *\ ] [ £= ] • • • [ *" ] of length n, provided that 
X = X1X2 ■ ■ • x n and y = 2/12/2 ■ ■ • 2M- Notice that a tape head of the 1TM scans two symbols a and r in the 
symbol [ " T ] simultaneously as a single symbol. This track notation can be further extended to the case where 
|x| ^ \y\. If |x| < |2/| and y = y x y 2 with \x\ = |yi|, then [ % ] denotes [ Jj ][ * lx ^ M ]; if |x| > \y\ and x = X1X2 
with |2/| = \xi\, then [ * ] denotes [ "j ][ # i4-is/i ], where # is a special symbol representing a "blank." 

For our later use, we also give a description of probabilistic finite automata. Here, we assume that all 
vectors are always expressed as row vectors. The notation M T for a matrix M denotes the transposed matrix 
of M. A matrix is called stochastic if every row of it sums up to exactly 1. A one-way (rational) probabilistic 
finite automaton (or lpfa, in short) M is a quintuple (Q, S, Vi n i, {M a } a£ ^, F), where Q is a finite set of inner 
states, S is an alphabet, Vi n i is an initial state vector with rational entries, each M a is a \Q\ x \Q\ stochastic 
matrix with rational entries, and F (C Q) is a set of final states. The set F induces a vector £p defined as 
follows: for each state q £ Q, the g-entry of £p has value 1 if q £ F, and otherwise. Without loss of generality, 
we can assume that Vi n i has always value 1 in its go -entry and in all the other entries. For each sequence 
X = o-\o~2 • • • o~ n in S n , M x is shorthand for M ai M l j 2 ■ ■ ■ M an . The acceptance (rejection, resp.) probability of 
M on input x € S* is defined as p acc (x) = v ini M^. x ^ (p rej (x) = 1 — p acc (x), resp.). 

3 Deterministic Computation with Advice 

We formally define the notion of (deterministic) advicqj and describe how to use it on one-tape linear-time 
Turing machines and finite automata. An advice function h is a function mapping N to T*, where T is 
a certain alphabet (which is particularly referred to as an advice alphabet). The advised language family 
1-DLIN/Zm (REG/n, resp.) is defined in [TU] as the collection of all languages S over alphabets S such 
that there exist an advice alphabet T, an advice function h : N — > T* , and a linear-time 1DTM (a ldfa, 
resp.) M satisfying the following conditions: (i) there are two constants c, d > such that, for every length 
n E N, \h(n)\ < en + d (\h(n)\ = n, resp.) and (ii) for every string x £ £*, X £ S iff M accepts [ h( J c X | ) ] 
(notationally, M([ h( ^ l) }) = S(x)). Note that REG/n contains non-regular languages, for instance, the 



•fin the literature, there are at least two different formulations of "advice" for the model of one-tape machines: Damm and 
Holzer's 2 and Tadaki, Yamakami, and Lin's I10| . These definitions are, however, computationally equivalent for, e.g., polyno- 
mial time-bounded computations. Theorem l3.2| gives an implicit justification for the choice of our advice model. 



language L eq — {O n l™ | n G N}. Surprisingly, the above two advised families, 1-DLIN/^n and REG/n, 
coincide [10, Proposition 4.11]. 

Lemma 3.1 10 1-DLIN/Zm = REG/n. 

In a polynomial-time setting, Karp-Lipton's advice naturally induces non-uniform computations. Similarly, 
languages in REG/n can be characterized in a certain non-uniform fashion. Here, we present a simple form 
of such non-uniform characterization of every language in REG/n. 

Theorem 3.2 For any language S over an alphabet E, the following two statements are equivalent. Let 

A = {(x,n) e E* xN | \x\ < n}. 

1. S is in REG/n. 

2. There is an equivalence relation =s over A such that 

(a) the total number of equivalence classes in A/ =5 is finite, and 

(b) for any length n G N and any two strings x,y G E* with \x\ = \y\ < n, the following holds: 
(x,n) =$ (y,n) iff, for all z with \xz\ — n, S{xz) — S(yz). 

Proof. (1 => 2) Assume that S is a language in REG/n over an alphabet E. Take a ldfa M = 

(Q,A,qQ,S,Q acc ,Q re j), an advice alphabet T, and an advice function h : N — >• T* satisfying that |/i(n)| = n 
and S = {x G E* | M accepts [ h( ^\) ]}, where A = {[ ° ] ireS,T6 T}. Without loss of generality, we can 
assume that \Q aC c\ — \Qrej\ — 1- This is possible by forcing M to enter a unique accepting/rejecting state 
after scanning the right endmarker $. For each length n G N and each string x G E*, we define q n , x as an 
inner state q for which (i) if \x\ < n, then M enters q just after reading <(:[ * ], where w is the first \x\ symbols 
of h(n), and (ii) if \x\ = n, then M enters q after reading <(:[ h( x n) ]$. Now, let us define the desired relation =5 
as follows: (2, n) = s (y, m) iff q n , x = q m ^ y . 

Condition (a) follows from the fact that |A/ =5 | = \{q m .y rn G N, y G E*}| < \Q\. Next, we want to 
show Condition (b). Consider two inputs [ * z v ] and [ ^ ] with |x| = \y\ = \w\, \xz\ = n, and h(n) = wv. 
Assume that (x, n) =5 (y, n). This means that, after reading §[ * ] as well as ((;[ ^ ], M enters the same inner 
state, say, q. Hence, even if the two inputs [ Zl } an d [ Zl ] are different, M behaves exactly in the same 
way during reading [ I ]$. As a consequence, for any x,y,n with \x\ = \y\ < n, q rhX = q rhy iff, for any z with 
\xz\ = n, there exists a halting state q' in Q acc U Q re j satisfying that q n ^ xz = q n . y z — q' ■ Condition (b) then 
follows immediately. 

(2 =>■ 1) To make our proof simple, we ignore the empty string and consider only the set A — {A}. Given 
a language S, an equivalence relation =s is assumed to satisfy Conditions (a)-(6). We aim at showing that S 
belongs to REG/n. By Condition (a), let A/=s = {Ai,A 2 , . . . , Ad} for a certain constant d G N+. For these 
equivalence classes A q in A/ =5, the following two properties hold. 

Claim 1 Let n G N + ; a G E, q G [d], anrf x,y G E* wit/i \x\ = \y\ < n. 

1. If (x, n), (y, n) (^ A q , then there exists a unique index q' G [d] such that (xa, n), (ytr, n) G A 9 ' . 

^. There exist two different indices, say, q a c C and q™J- in [d] satisfying that {(z, n) | \z\ ~ n, z G S} G A <„> 
and {(z, n) | |z| = n, z g" S*} C A <„> . 

Proof. (1) Let x and y satisfy |a;| = \y\ < n. We want to show that (x,n) =5 (y,n) implies (xa,n) =5 
(ya,n). Note that the "uniqueness" requirement follows from the fact that A qi n A q2 = for any distinct 
indices 01,02 G [d]. Toward a contradiction, we assume that {x, n) =s (y,n) but (xo~, n) ^5 (yo~,n). Take any 
string 2 with \z\ = n — \x\ — 1. Notice that such a z exists since |x| < n — 1. First, consider the case where 
xaz G S. Since (xcr, n) ^s {yo~, n), by Condition (b), we have j/trz <£ S, which further implies (x,n) ^s {v, 71 )- 
This is obviously a contradiction against our assumption. We therefore conclude that (xo~, n) =5 {ya, n). The 
case where xaz $ S is similar. 

(2) Assume that \x\ = \y\ = n. As a special case of Condition (0), it follows that (*) (x, n) =5 (y,n) iff 
S(x) = S(y). In particular, if x,y G 5, then (x,ri) =s (y,n). Therefore, there exists an index e G [d] such 
that {(a;, n) | \x\ — n, a; G S} C A e . We write this e as gi"c- Likewise, we have {(x, n) | \x\ = n, x G' S 1 } C A e / 



for another index e' E [d\. By Statement (*), e^e' follows immediately. Write this e' as q^L and we then 
obtain the claim. D 

Let us return to the proof of the theorem. For any length n E N + and any index i E [n] , we define a series 
of finite functions h n ,i '■ [d] X £ — ► [d] U [d] 3 as follows. 

(i) Let h ni i(q, a) = q' if (a, n) G A q i. 

(ii) For any index i with 1 < i < n, let h n ,i(q,o~) — q' if there exists a string x with |x| = i — 1 such that 
(x,n) E A 9 and (x<7, n) E A q /. 

(iii) Let h ny7l (q,a) = (q' , qacc, q^j) if there exists a string x with |x| = n — 1 such that (x, n) E A q and 
(x<7, n) E A q '. 

Hereafter, we treat each h n ,i as a new symbol and define Y = {h nt i \ n > l,i E [n]}. It is important to note 
that r is a finite set. Our advice string h n of length n is defined to be h nt ih n> 2 ■ ■ ■ h rhn . 

At this point, we need to show that h n> i is indeed a function. Consider the case where 1 < i < n. 
Assuming that h n> i(q, a) = gi and h n< i(q, a) = q%, we take two strings xi and X2 of length < n satisfying that 
(xi,n), (x2,n) E A q , (xi<j, n) E A qi , and (x2<7, n.) E A 92 . The uniqueness condition of Claim [TJl) then yields 
the desired equality q\ = q2- The other cases for h n< i and h n ,n can be similarly treated. 

Now, let us define a finite automaton M with its transition function 5 as follows. We prepare four 
new inner states g acc , qrej, Qaca an d q re j' which do not appear in Q. Let 6(qa,$) — go and, moreover, let 
<%) I K,i ]) = h nt i(q,a) for every index i € [n - 1]. When h n . n (q,cr) = (q', g4™c, g;™j) , let c5(g, [ h £ n ]) = g^ cc 
if g' = gi"c; let <5(g, [ h ^„ ]) = q' rej if g' = g;"j. Finally, let <5(g„ cc , $) = q acc and <5(^ ej , $) = q rej . 

In this end, we want to show that x E S iff M accepts [ h * n ]. Let x = <7i (T2 •• -an and assume that 
(A, n) E A go , (<Ti,n) E A qi , (cricr2,n) E A ?2 , . . ., (x,n) E A ?n . First, let us consider the case where x E S. 
From now, we intend to prove by induction that 

(**) qi = 5(qoM nZ'.'.Z,, ]) for every i£ [0,n-I] z , 

where S is the extended transition function induced from S. The basis case i — holds since S(qo,i^) = go- 
From the induction hypothesis on i < n — 1, it follows that 

HqoM C ::: v^ ]) = W».^;!-:JM^J) 

= h n ,i + i(5(qo,b[ hZl'-'-'-hl,, ]),ctj+i) 
= h n ,i+i{qi, o"i+i) = gi+i. 



Thus, Statement (**) holds. In particular, we have q n -i = <5(go,<(i[ h °\ ...l" _1 _i ])• Note that, by the definition 

" } ). Since x E 5, Claim QJ2) implies A„ n = A ( 



of /i Tli „, hn t n(q n -i)0'n) = (<?td ^icc, g^™,-)- Since x E 5, Claim[Tj2) implies A qn = Aw, from which we obtain 



and thus 6(qo,fy[ h " ]$) = g acc - This means that M accepts [ h x n ]. The other case x ^ 5 is similar to the 
previous case, since the only difference is the final step of the above argument. This completes the proof of 
Theorem O □ 



4 Probabilistic Computation with Advice 

Probabilistic computation has been a useful tool for designing many practical algorithms. We shall move our 
attention to linear-time IPTMs, supplemented with deterministic advice. In a similar fashion to 1-DLIN/Zm, 
we define two families 1-PLIN/Zm and 1-C=LIN/Zm by simply modifying the definition of 1-DLIN/Zm using 
IPTMs in lieu of lDTMs. 

Earlier, Tadaki et al. [1(3 showed that CFL n 1-C=LIN £ REG/n. From their result, since 
REG/ra C l-C = LYN/lin, it immediately follows that REG/n is properly included in 1-C = LIN/Zm (as well 
as co-l-C=LIN//m). 



Proposition 4.1 REG/n g l-C=LIN/im n co-l-C=LIN/Zm. 

The above proposition indicates that, even in the presence of advice, probabilistic computation is much 
more powerful than deterministic computation. Naturally, we can question how powerful the families 1-C=LIN 
and 1-PLIN are when deterministic advice is allowed. We shall provide several answers to this question. Our 
first answer is the following. 

Theorem 4.2 1-C=LIN/Zm ^ co-l-C=LIN/Zm ^ 1-PLIN/Zin. 

To prove this theorem, we need a key lemma, which gives a new criterion that every language in 
1-C=LTN/Zm must satisfy. Historically, in early 1970s, Dieu [3] showed the following criterion (in our ter- 
minology): if L is in 1-C=LIN, then there exists a number m £ N + such that, for any u,y,v £ S*, 
{uy l v I i £ [0, m— l]z} C L implies {uy l v \ i £ N} C L. Unfortunately, his criterion cannot be extended to our 
advised language family 1-C = LTN/Zin, because advice strings may change as input size increases. To prove 
our theorem, we should seek another criterion for 1-C=LIN/Zm. The next lemma provides one such criterion. 

Lemma 4.3 Let A £ 1-C=LIN/Zm be any language over an alphabet E. There exists a positive integer m 
that satisfies the following statement. Let n,£ £ N and z £ E* satisfy that n > 1, £ < n — 1, and \z\ — £. Let 
A n . z = {w £ E™ - ^ I wz £ A}. There exists a subset S C A n>z with \S\ < m such that, for each string y £ E , 
if {wy I w £ S} C A then {xy \ x £ A n ^ z } C A. 

Before proving Lemma 14.31 we shall present the proof of Theorem 14.21 This proof exemplifies usefulness 
of the criterion given in the lemma. 

Proof of Theorem 14.21 Let E = {0,1} and consider the language Dup — {ww | w £ E*} (duplicated 
strings). Later, in the proof of Proposition 15.41 we shall prove that Dup £ 1-C = LIN/Zm. It is therefore 
sufficient to show below that Dup £" l-C = LIN//m. 

For simplicity, we write A for Dup. Now, we want to show that A £" 1-C—LIN/lin. To lead to a contra- 
diction, we assume that A £ 1-C=LIN/Zm. By Lemma 14.31 there is a positive integer m that satisfies the 
conclusion of the lemma. Now, we choose the minimal even integer n such that 2™/ 2 — 1 > m + 1. Define 
z = l™/ 2 . Clearly, the set A n , z (= {w £ E™/ 2 | wz £ A}) satisfies that A rhZ = E"/ 2 \ {z}. There exists a set 
<S C yl„ jZ with |5| < to that satisfies the lemma. Take any string y in E"/ 2 \(SU {2}). Note that y £ A„. 2 
because A nj2 = E™/ 2 \ {0} and y ^ z. Since wy £ A for any string w £ S, the lemma implies that yy £ A. 
This is a contradiction against the fact that yy Z?up = A. Thus, we conclude that A £ 1-C = LIN/Zm. As a 
result, we obtain the desired separation: 1-C = LIN/Zm 7^ co-l-C = LIN/Zm. 

Next, we assume that co-l-C = LIN/Zm = 1-PLIN/Zm. Similar to 1-PLIN, the advised family 1-PLIN/Zin 
is closed under complementation; that is, 1-PLIN/Zin = co-1-PLIN/Zm. This closure property implies that 
1-C = LIN/Zm is also closed under complementation; however, this contradicts the above separation result 
between 1-C = LIN/Zin and co-l-C = LIN/Zm. It therefore follows that co-l-C = LIN/Zin ^ 1-PLIN/Zm, as re- 
quested. □ 



We shall present the proof of Lemma 14.31 For this proof, we shall use the following characterization of 
1-C = LIN/Zzn, described in terms of lpfa's and advice functions. This characterization is in essence analogous 
to Lemma 13.11 Since it can be shown by an argument similar to the proof of Claim [3] (which uses a notion of 
"folding machine" in (TU]), we omit its proof for readability. 

Lemma 4-4 F° r an V language A over an alphabet E, A £ 1-C = LIN/Zm iff there exist a lpfa M and an advice 
function h that satisfy the following: for every string x £ E*, x £ A iff M accepts [ h ^ n ] with probability 
exactly 1/2. 

The following proof of Lemma 14.31 exploits a fundamental property of a stochastic matrix, which states 
that each of its rows sums up to 1 . 

Proof of Lemma 14.31 Let A £ 1-C=LIN/Zm be any language over an alphabet E. Lemma T4.4I guarantees 



the existence of a lpfa M — (Q, A, Vi n i 1 {M cr } a£ ^ F), an advice alphabet T, and an advice function h : N — > T* 
such that, for every string x G E*, x G A iff Prob M [M([ h ^ n ]) = 1] = 1/2, where A = {[ " T ] | a € E, r e T}. 
Recall from Section [2] our assumption on i/j n j and £p. 

We set m = \Q\ and choose n,f(;N and z G E^ arbitrarily. Since the lemma trivially holds for I = 0, let 
us assume that £ > 0. For simplicity, let /i(n) = rs for two strings r G E™~^ and s G E . Now, we focus our 
attention to the set A njZ = {w G E™ | raz G A}. Notice that the lemma is also true by setting S — A n>z 
when \A n ^ z \ < \Q\. It thus suffices to consider the case where \A n , z \ > \Q\. 

Henceforth, we write w = [ ™ ], x = [ I ], z = [ * ], and y = [ I ] for notational convenience. Note that 
the acceptance probability p aC c(wz) =def v iniMkwMz$£iF (note that this notation suppresses the advice string 
rs) equals 1/2 for all strings wz in A. Choose a maximal subset S' of linearly-independent vectors (which form 
a set of basis vectors) in the set V — {vtntM^ \ w G A rijZ }. Clearly, since each vector ^wMte has dimension 
\Q\, there are at most \Q\ linearly-independent vectors in V. This implies that \S'\ < \Q\ = m. The desired 
set S is now defined as S = {w G A n ^ z \ ViniM^w G S'}. Note that any vector Vi n iM^ in V — S' can be written 
as a linear combination of basis vectors in S': 

(*) %.% = 2_, aw ' ViniM^, 

wES 

where {a w } w ^s is a set of appropriate real numbers. As a key claim, we show the following statement. 
Claim 2 J2 w es a ™ = 1 - 

Proof. For our convenience, let y,„,M(,j = (a x ,i)ie[m] an d let VmiM^ = (a w ,i)jg[ m ] for each element w G S. 
From Equation (*), we obtain (a x> i) iE [ m ] = (J2 w es a w a w,i) iG [ m ] ■ For every string w G S, since M^ is a 
stochastic matrix, it holds that ^._ 1 a w ^ = 1. Likewise, we have J3 i=1 a^, = 1. From these equations, we 
obtain J^TLi {J2wes awa " w - i ) ~ •"•> which further implies 



/ i / C^uj^u;,i I — / . *^w I / Q>w.i I — / Ck?j 



?'— 1 V^ES 1 / ^ES 1 \i— 1 / w£S 

This finishes the proof of the claim. □ 

At last, we shall show the desired property of S: for every string y G E , {«jy | w G 5} C A implies 
{xy | x G A ll:Z } C A. To show this property, let y be any string in E £ and assume that wy G A for all strings 
w in S. From Claim [21 since p a cc(wy) = 1/2, it follows by Equation (*) that, for each string x G A nyZ , 

Pacc(xy) = ViniM^My^p = I ^ q m • I/iniM^tf J M S$ ^J 

= ^ a m (yjnjM^M^^) = 2J a ™ -Pacc(wy) 

wes wes 

1 



IE 



2 ^ a » - 2 ' 

Hence, we obtain xy G A. Because x is arbitrary in A nyZl we finally obtain the desired property and therefore 
the lemma. □ 

The advised language family CFL/n was introduced in |12j . analogous to REG/n, using one-way nonde- 
terministic pushdown automate^} (or lnpda's, in short) together with deterministic advice whose length equals 
input size. A class separation between CFL/n and l-C = LIN/^in is also possible. 

Proposition 4.5 1-C=LIN <£ CFL/n. Thus, 1-C=LIN/Zm <£ CFL/n. 



§ Roughly speaking, a lnpda is a one-way nondetcrministic automaton equipped with a stack, which is an additional read/write 
tape whose access is regulated by the first-in, last-out policy. 



Proof. Let Eg = {ai, 02, . . . , ag, #} be our alphabet. Consider the language Equals composed of all strings 
w over Eg such that # a (w) = 4^a'(w) for any pair a,a' £ Ej-ff}. Since Equals sits outside of CFL/n |12) . 
it is enough to show that Equals belongs to 1-C = LIN. For any two fixed indices i, j G [l,6]z, we denote by 
Lij the language {w £ Eg # Qi (to) = # Qj (w)}. It is not difficult to show that each language L i: j belongs 
to 1-C = LIN. Moreover, note that Equals — f] i=2 £i,i< A crucial point is that 1-C = LIN is closed under 
intersection [10]. This closure property implies that Equals belongs to 1-C = LIN. □ 

As an immediate consequence of Proposition 14.51 we obtain 1-PLIN/Zm ^ CFL/n. In the following, we 
shall prove the other direction: CFL/n <£. 1-PLIN/Zm. Notice that the non-advice separation CFL (/ 1-PLIN 
of Nasu and Honda [7] (see also [10, Proposition 6.7]) does not imply our desired separation. 

Theorem 4.6 CFL <£ 1-PLIN/Zm. Thus, CFL/n £ 1-PLIN/Zm. 

This theorem follows from the next lemma, which gives a new criterion for languages in 1-PLIN/Zm. This 
lemma sharply contrasts with Lemma 14.31 

Lemma 4.1 Let A G 1-PLIN/Zm over an alphabet E. There exists a positive constant m that satisfies 
the following statement. Let n,£ G N be arbitrary numbers with n > 1 and £ < n — 1. There exists a set 
S = {wi, W2, ■ ■ • , w m } C E"~^ with \S\ = m for which the following implication holds: for any set T C E f , if 
\{A(wiy)A(w2y) ■ ■ ■ A{w m y) \ y G T}\ > 2 m , then it follows that, for any string x G E™ ; there exists a pair 
y,y' G T such that xy G A and xy' g - A. 



From this lemma, Theorem 14.61 easily follows. In the following proof of the theorem, the notation denotes 
the bitwise binary inner product. 

Proof of Theorem 14.61 Let E = {0, 1} for simplicity and consider the language IP* = {axy | a G 
{A,0, l},x,y G E*,|x| = \y\,x R y = (mod 2)}. Since IP* is in CFL 13], we want to show that 
IP* & 1-PUN/lin. 

Now, assuming that IP* G 1-PLIN/Zm, we take a positive constant m that satisfies Lemma [4771 Choose a 
sufficiently large even number n and let £ — n/2. There exists a subset S = {wi, W2, ■ ■ ■ , w m } of E with m 
distinct elements that satisfy the lemma. To each binary sequence r — (r±, r%, . . . , r m ) (seen as a string) in 
E m , we assign a certain string y r G E n ~ f satisfying that wf y r = ri (mod 2) for all indices i G [m\. Note 
that y r 7^ y r i for any distinct pair r,r'. By collecting all such y r 's, we define T — {y r \ r G E m }. Clearly, 
\T\ = 2 m and therefore \{IP*{w x y) ■ ■■IP*{w m y) \ y G T}\ = \T\ > 2 m . Since n is sufficiently larger than 2 m , 
there is a string i£S' satisfying x R y r = (mod 2) for every sequence r in E m . In other words, xy G IP* 
holds for any y G T. For this x, the lemma yields a pair y, y' G T for which xy G IP* and xy' ^ IP*. This is 
a contradiction against the choice of x. Hence, we conclude that IP* $ 1-PLTN/Zm. □ 

Finally, we shall give the proof of Lemma 14.71 This proof relies on, similar to Lemma 14.41 a new lpfa- 
characterization of 1-PLIN/Zm. 

Lemma 4.8 For any language A over an alphabet E, A G 1-PLIN/Zm iff there exist a lpfa M and an advice 
function h satisfying the following: for every input x G E*. x G A iff M accepts [ ft( x n) ] with probability more 
than 1/2. 

Proof of Lemma 14.71 Let A be any language in 1-PLIN/Zm over an alphabet E. For this A, we can 
take a lpfa M, an advice alphabet T, and an advice function h : N — 5- T*, as described in Lemma [4.81 Let 
AI = (Q,A,v ini ,{M (J } ae jy,F), where A = {["]|ctgE,tG T}. To make our argument simple, we make 
the following extra requirement: the success probability of M on any input string never becomes exactly 1/2. 
This requirement can be easily met by an appropriate modification of the given lpfa M (see, e.g., |10|). In 
the subsequent argument, we assume that Probjvf[-^([ h( n ) ]) = A(x)] > 1/2 for every string x. 

Choose n,£ > 1 arbitrarily with I < n — 1. To follow the proof of Lemma T4. 31 we intend to use the same 
notations x = [ I ], y — [ y s ], w = [ ™ ], and Wi = [ w j ] for an advice string h(n) — rs. The only difference 
from the proof of Lemma |4. 31 is that we do not need to take a fixed input string z that forces M to accept. 



Here, we choose a maximal subset S' of linearly-independent vectors in the set {^wMte I w G S™ e } and we 
then define S = {w G £"~ £ | i/,™ A/^ G S"}. Set m = |5| and let S = {w x , w 2 , ■ . . , w m }- Now, fix x G £ n ^ 
arbitrarily. Claim [5] ensures the existence of a series {aa,Ji £ [ m ] of real numbers that satisfy (i) $^ =1 a«, ; = 1 
and (ii) i/ in iM^ = XT=i a ™* ( v imM^.) . 

Next, let T be an arbitrary subset of T, e and assume that, for any binary series r = (ri,r2, • • • ,r m ) G 
{0, l} m , there exists a string y r G T satisfying A{wiy r )A{w2y r ) • • • ^4(w m y r ) = r. For each string y G S £ and 
each index i G [to], let p acc {wiy) = 1/2 + /3^ a (as before, this notation suppresses the advice string rs) for a 
certain real number Pi :V G [—1/2, 1/2] — {0}. For such y's, we have 

Pacc(xy) = E a ™* (viniM^ z My $ £p) = E a w,Pacc(w t y) 

= 5+£«™Av - ^+£(-i) 1 - A( ™ iW) lft, ! ,l<w 

To complete the proof, we want to define a binary series r = (ri,...,r m ) as follows: let r, = (or 
equivalently A(wiy r ) — 0) if a Wi < 0, and let n — 1 (or A(wiy r ) = 1) if a Wi > 0. Let us consider the string 
j/ r associated with this r, and make y r as our desired string y. By the choice of y, we have 

E (-l) 1 -^^;,^ = E IMK.I > °' 

which implies p a cc(xy) > 1/2. This means that xy G A. Next, we define r' as the bitwise negation of r and 
write y' for y r >. Similar to the previous case, we have 



E (-i^^i/vk. = E (-i)i/3^n^i < o, 

and thus we obtain xy' G" A. This completes the proof of the lemma. 



5 Power of Randomized Advice 

We have so far discussed the roles of a single advice string given per each input length n. Instead of giving 
such a deterministic string, we can provide "randomized" advice strings, each of which is produced according 
to a certain fixed probability distribution. It turns out that such randomized advice often endows an enormous 
power to its underlying machine's language recognition. 

In this paper, randomized advice refers to a probability ensemble {D n } ng N, in which each probability 
distribution D n is defined over all advice strings of length n. Let to G N, let x be any input string, and let 
D m be any probability distribution over T m , where T is an advice alphabet. The notation [ £ ) indicates a 
random variable that expresses [ ™ ], which is chosen randomly with probability D m (y) over all strings y in 
T m . Similarly, given a machine M, the notation M([ o m ]) denotes a random variable expressing the outcome 
M([ I ]) of M on an input [ * ], which is chosen randomly according to D m . 

We use the notation 1-BPLIN/fflm to denote the collection of all languages L for which there exist a 
linear-time 1PTM M, an error bound e G [0, 1/2), a probability ensemble {D n } n ^, and a linearly-bounded 
function £ : N — >• N satisfying: for every input x, if x G L then M accepts [ D x m ] with probability at 
least 1 — e; otherwise, M rejects [ D x m ] with probability at least 1 — e, where to = £(n). For brevity, 
we write ProbM,£) m [M ([ o m ]) = L(x)] > 1 — e. In a similar fashion, we may define l-C = LIN/i?/m and 
l-PLIN/_RHn with linear-size randomized advice; however, those two language families are so powerful that 
they can recognize all languages. 

Proposition 5.1 The advised language family l-C = LIN/i?/m as well as l-PLIN/i?Zm consists of all lan- 
guages. 

Proof. Let A be an arbitrary language over an alphabet S. Our goal is to show that A belongs to 
1-C = LIN / Rlin. To simplify our proof, we assume that X ^ A and we hereafter consider only positive input 



lengths. For each length n G N+, we write A n for A n E n and A„ for E™ \ A n . Moreover, we set P = E U {#}, 
where =#= is a special symbol not in E. 

Let us define a probability distribution D n over T", which generates only strings in E n as well as the string 
#" with positive probabilities. Let y be any advice string in F™. Whenever y £ E™ U {#"}, let D n (y) = 0. 
Henceforth, we are focused only on advice strings y in E™ U {#"}■ If A n — 0, then we set D n (y) = 1 if 
y = #", and D n {y) = for the other advice strings y. When A n ^ 0, we set D n (y) = l/\A n \ if y G v4„, and 
D n (y) = if y G A„ U {#"}■ Our lpfa M works as follows. Let [ * ] be an arbitrary input of length n. If 
x = s, then M accepts [ * } with certainty; otherwise, it accepts and rejects [ * ] with an equal probability. 
Note that, when s = jf l . [ * ] is never accepted. 

In the case where x G" A, the acceptance probability of M on [ * ] is at least l/2+D n (x)/2 (= l/2+l/2|A n |), 
which is clearly more than 1/2. By contrast, when i£i, the acceptance probability on [ " ] is exactly 1/2 since 
D n (x) = 0. Therefore, A belongs to 1-C=LIN/ Rlin. Since l-C = LIN/i?Zm C 1-PLIN/ Rlin, the proposition 
easily follows. □ 

We return to the advised language family l-BPLIN/i?Zm. Earlier, Tadaki et al. [TU] showed that 1-BPLIN 
coincides with REG. Let us present a similar characterization of 1-BPLIN / Rlin using one-way finite automata. 
First, we introduce REG/i?n — a natural extension of REG by supplying randomized advice. Formally, a 
language A over an alphabet E is in REG/i?n if there exist a ldfa M, an error bound e G [0, 1/2), an advice 
alphabet T, and a probability ensemble {D„}„ 6 n over T* that satisfy the following condition: for any length 
n G N and any string x G E", if x G A then M accepts [ „ n ] with probability > 1 — e; otherwise, M rejects 
[ D * n ] with probability > 1 — e. Likewise, we can define another advised language family CFL/i?n using 
lnpda's (instead of ldfa's) together with randomized advice. Obviously, REG/i?n C CFL/ Rn. In comparison 
to Lemma [3. 1[ we shall prove that the two families l-BPLIN/i?Zm and KEG /Rn coincide. Notice that this 
result is not a direct consequence of the aforementioned equality 1-BPLIN = REG of Tadaki et al. [TU]; rather, 
it is from the fact that bounded-error probabilistic computation can be integrated into randomized advice. 

Theorem 5.2 1-BPLIN /Rlin = KEG/Rn. 

Proof. Since every lpfa can be simulated by a certain linear-time 1PTM, the inclusion REG/i?n C 

l-BPLIN/i?Zm follows immediately. Hereafter, we pay our attention to the remaining inclusion. The following 
proof consists of three stages. Assume that A is any language in 1-BPLIN / Rlin, witnessed by a linear-time 
1PTM M and a probability ensemble {-D„}„eN over the set I* of advice strings. To ease the notational 
complexity, we use the same terminology given in [10J . In the first stage, we claim that this linear-time 1PTM 
M can be replaced by a certain lpfa N even in the presence of randomized advice. 

Claim 3 There exists a lpfa N and an error bound s G [0, 1/2) such that, for every length n G N and every 
string x G E™ 7 PTob N ,D n [N([ £ n ]) = A(x)\ > 1 - e. 

Proof. From the length requirement for our randomized advice {Z?„}„ e N, we can assume without loss of 
generality that, for every string x G E*, ProbM,D m ( ia ,n [M{[ « m n x i) ]) = A(x)] > 1 — e, where a length function 
m satisfies that n < m(n) < kn for every length n G N. Now, we assume that M takes an input of the form 
[ a! * l ™'~™ ] with an advice string w G r m ("). 

How can we simulate M's moves along the tape cells indexed from —2k(n — 1) to 2k(n — 1) — 1 using 
only its "input area" (i.e., the tape region where the original input string x is written, together with the 
two endmarkers)? As shown in [10, Section 4.2], it is possible to "fold" M's tape content into its input area; 
namely, the tape content is partitioned into 4fc blocks of size n — 1, indexed from left to right by numbers 
between — 2k and 2k — 1. Each block's content is written in one of 4fc tracks, indexed from top to bottom by 
the same numbers, of a new tape so that we use only n tape cells to simulate M's entire behavior on the input 
j m# » j_ This gives rise to a so-called "folding machine," which simulates M using only its input area. 

Let cont(x, w) denote a string obtained by folding the tape content [ x# ™ " ] into its input area. By 
deleting all symbols in E from cont(x, w), we obtain a new advice string, say, w' of length n. For this w' , set 
D' n {w') = D m t n \(w). A new 1PTM M' behaves as follows: on input [ //, ], first recover the string cont(x,w) 
in linear time and then simulate M's folding machine using cont(x,w) as its new input string. 

From the above argument, M' takes new advice that is randomly distributed over strings of length equal 
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to input size and moves its tape head between (and on) the two endmarkcrs. Apply [10, Lemma 6.5] to obtain 
a rational one-way generalized probabilistic finite automaton (or 1GPFA, in short) N such that Pm'{[ Z ]) = 
Pn([ w ]) f° r an y P a i r x,w. We then simulate this 1GPFA by another rational lpfa N with preserving the 
same acceptance/rejection probability. This proves the claim. □ 

Now, we assume that N is a lpfa working with the randomized advice {_D^}„ e N- Let N — 
(Q, A, Ui n i, {Mff}^, F), where A = {[ ° ] er G £, r G L}. Notice that each M a uses only rational transition 
probabilities. In the second stage, we modify M a so that it uses only transition probabilities of either or 1/d 
for a certain fixed positive integer d. This modification can be done in the following way. Choose a sufficiently 
large positive integer d so that any transition probability of N can be expressed as k/d, where k G N. Here, we 
wish to define another lpfa N' . Assume that Q = {gi, q2, ■ ■ . , q m }- For each inner state qi, we prepare d new 
states {(ft , <?2 , ■ • ■ >Qd }■ ^ n scanning [ " T ], if there is a transition from inner state qi to qj with transition 
probability pij/d, then, for each index k G [d], we assign the probability 1/d to a new transition from q£ to 

q\ for every index I G [Pi,j\] we assign the probability to a transition from q£' to q\ for any other indices 
I. It is not difficult to show that this lpfa N' has the same acceptance/rejection probability as the original 
lpfa N. 

Let N' = (Q',A,i'i n i,{N (T } ae ^,F'). For simplicity, we assume that all inner states in Q' are enumerated 
in a pre-fixed total order. In this final stage, we want to define the desired ldfa M and the desired probability 
ensemble {D n } ne ^ that together simulate N' using {_D^}„ s n with bounded-error probability. Let A = {(r, k) \ 
t G L, k G [d]} be our new advice alphabet. For any advice string w — (n, fci)(r 2 , k 2 ) ■ ■ ■ (r n , fc„) G A" of length 
n, define D n (w) = D' n (TiT 2 ■ ■ ■ r n )/d n . Our ldfa M then behaves as follows. Note that, on scanning a symbol 
if = [ ° } in inner state qj, since N v is stochastic, N' enters exactly d different states, say, q'j X , q'j 2 , . . . , q' ■ d 
(in the given order) with probability exactly 1/d. Associated with this transition, we define a new transition 
of M on a new input symbol [ (T " fc) ], where k G [d]. In scanning this symbol [ {T ° k) ], M enters state </• k 
from qj deterministically. By the definitions of M and D ni it directly follows that the acceptance/rejection 
probability of M with {D n } nG ^ equals that of N' with {D^}„ g n. 

In conclusion, A is recognized by the ldfa M with the randomized advice {-D„}„ g N. This implies that A 
is in REG/n. Since A is arbitrary, we conclude that 1-BPL1N/ Rlin C REG/i?n. □ 

We have proven that l-C = LIN/i?/in and l-PLIN / Rlin are powerful enough to capture all languages. Even 
for weak families, such as REG, randomized advice is more resourceful than deterministic advice. 

Proposition 5.3 DCFL n REG/ Rn <£ REG/n. 

Proof. Consider a "marked" version of the language of even-length palindromes: Pal# = {w#w R | w G 
{0,1}*} defined over the ternary alphabet S = {0,1, #}. Clearly, Pal# belongs to DCFL (because of the 
presence of the center marker #). It is proven in [T2] that the language Pal = {ww R w G {0, 1}*} is outside 
of REG/n. By a similar proof, we can show that Pal# is not in REG/n. What remains is to prove that Pal# 
belongs to 1-BPLIN/fflin, which equals REG/i?n by Theorem IQ1 

Fix n arbitrarily. Our probability distribution D n chooses advice strings of the form y#y R , where y G 
{0, l} n , with an equal probability. More precisely, for each advice string y in {0, 1}", we define D n (y#y R ) = 
2~"; for any other advice string w in E 2n+1 , let D n (w) = 0. Next, we describe an underlying 1PTM M for 
Pal# with the above randomized advice {D n } ne ^. The machine M behaves on input [ * ] as follows. If 
\x\ is even, then M rejects the input. Assume that |x| = 2n + 1 and w = y#y R - Using this advice string 
w, M rejects the input if x is not of the form x\^x 2 for certain strings x\, x 2 G {0, 1}". Henceforth, let us 
assume that x = x\#xi with x\,x<i G {0, 1}™. The machine M then computes two values xiQy (mod 2) and 
X2 y R (mod 2) separately and M finally checks whether x\ y = X2 y R (mod 2). If those two values are 
equal, then M accepts the input; otherwise, it rejects. 

Obviously, if x\ = x R then M accepts [ * ] for any string y G {0, 1}™. Otherwise, by the property of 0, 
M accepts [ * ] for exactly a half of y's in {0, 1}™. Furthermore, if we modify M and {-D n }neN to run in 
parallel the above procedure twice with two randomly-chosen advice strings yi#y R and 2/2^2/^ (which can be 
given as a single advice string of the form [ vl * Vl R ]), then we can reduce the error probability down to 1/4. 
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It therefore holds that Pal # G l-EPLIN / Rlin. □ 

Proposition 5-4 KEG/Rn n 1-C=LIN/Zm <£ CFL/n. 

Proof. Recall the language Dup = {ww | w G £*} over the alphabet S = {0, 1}. An idea similar to 
the proof of Proposition 15.31 proves that Dup belongs to KEG/Rn. Since Dup g" CFL/n [12 , we obtain 
REG/i?n (/ CFL/n. Next, we shall show that Dup 6 1-C = LIN/Zm. Our advice function h for Dup marks the 
"center" of the tape; namely, h(n) = 0™/ 2_1 110"/ 2_1 if n is even, and h(n) = 10™ _1 if n is odd. 

Now, we want to define a 1PTM M for Dup with h. In the following description, O denotes an all-zero 
matrix of an appropriate size and / denotes an identity matrix. For convenience, we describe the behavior of 
M as a series of stochastic matrices {M a } ae ^, similar to a lpfa, defined on four inner states {qo, 9i,<?2, 93}- 
Here, we focus only on inputs of the form [ ™ { ™ n) ], where w = W\ ■ • ■ w n and w' — w[ ■ ■ ■ w' n in E 2n . Initially, 
Mi, changes qo to qi with certainty. While reading a symbol a 6 {0, 1} appearing in the first half part of the 
string ww' , M applies the following matrices: 

-^0 = I A " ) an d Mi = I J , where 



( A 




°\ 


\o 




1 ) 


1 











1 





1/2 





1/2 




and B = 

After scanning symbols in w, M reaches the inner states qo, q\, and 92 with probabilities po = Yli-w =0 ^ 

Pi = Y,i-. Wi =i 2 ~\ and 2_ ™> respectively. 

When the head reaches the middle of 11 written in the lower track, it applies the following matrix: 

M midd i e = I I , where 

/ 2 -(„-i) 1 _ 2 -(n-2) \ , n _/2-(™- 1 ) l-2-("- 2 ) 

6 - I 2~(«- 1 ) 1 - 2-(™- 2 ) J ~ I 1 

This matrix makes the probabilities of entering states go , <7i, and 92 equal to po2 _ ^™ _1 \ Pi2 _ ^" _1 \ and2 _ ' n_1 ^, 
respectively. (The above matrix can be realized by the following head move: the head moves back to the left 
endmarker and returns to the end of the first half section by flipping fair coins.) 

As each symbol a in the second half of the string ww' , M applies the matrices M' = M\ and M[ = Mo, and 
after reading w', M enters qo and q\ with extra probabilities of 2~(™~ 1 ) Y^i- w '=o ^ % and 2~ ( "~ 1 ) Yli-w'=i ^~ % i 
respectively. 

Finally, on scanning $, if M is already in qo and in q\, it respectively enters a rejecting state, say, q re j and 
an accepting state, say, q acc with certainty; otherwise, it enters q acc and q re j with an equal probability. 

It is not difficult to show that w = w' iff the probability of reaching q acc is exactly 1/2. Therefore, Dup 
belongs to l-C=LIN//m. □ 

Since REG/n C CFL/n, Propositions 15.31 and 15.41 both yield a class separation REG/n C REG/i?n. 
Proposition 15.41 also yields another separation between CFL/n and CFL/i?n, because obviously REG/i?n C 
CFL/i?n. 

Corollary 5.5 CFL/n g CFL/i?n. 

In the proof of Proposition 15.41 we have shown that Dup belongs to REG/i?n. This fact helps us prove 
the following class separation as well. 

Proposition 5.6 REG/i?n <£ l-C = LIN//m U co-l-C = LIN/Zm. 

Proof. Since Dup is in REG/i?n (from the proof of Proposition 15.41) and REG/i?n is obviously closed 
under complementation, Dup is also in REG/i?n. In the proof of Theorem 14.21 however, it is shown that Dup 
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does not belong to 1-C = LIN/Zm. Those two results imply that KEG/Rn <i 1-C = LIN/Zm. By considering 
their complement classes, we obtain another separation: co-REG/i?n ^ co-l-C = LIN/im. It therefore follows 
that REG/i?n = co-REG/i?n </ co-l-C = LIN/^n, as requested. □ 



6 Limitation of Randomized Advice 



The previous section has demonstrated a power of randomized advice; for example, we have shown that 
REG/i?n <£ CFL/n U TC = LIN/7m. By contrast, this section shall discuss a limitation of the randomized 
advice. In particular, we intend to show that CFL ^ REG/i?n; in short, even with a help of the randomized 
advice, REG cannot capture CFL. This result significantly extends the previously-known separation CFL ^ 
REG/n [TO] 

Theorem 6.1 CFL <£ REG/i?n. 

From this theorem, we can deduce that GEE/ Rn properly contains REG/i?n because, otherwise, CFL/n 
should be included in KEG/Rn, contradicting the theorem. 

Corollary 6.2 REG/i?n g CFL/i?n. 

Henceforth, we shall prove Theorem 16.11 In order to do so, we borrow an idea from communication 
complexity theory because our model of randomized advice is loosely related to a model of two-party one-way 
communication with shared randomness. First, we introduce a new complexity class Aver-REG/n. (As for an 
introduction to average-case computational complexity theory, the reader may refer to |llj for instance.) The 
class Aver-REG/n consists of all distributional problems (A, fi), where A is a language over an alphabet E and 
t 1 = { n n}neN is a probability ensemble over E*, such that there exist a ldfa M, an advice function h, and an 
error bound e E [0, 1/2) satisfying the following condition: for every length n E N, Prob^^ [M ([ h * n) ]) = 
^(^O] > 1 — £j where "x ~ /i n " means that x is chosen randomly according to u n . 

Proposition 6.3 If A belongs to KEG/Rn, then {A, a) is in Aver-REG/n for any probability ensemble a. 

Recall that our goal is to present a context-free language A that does not belong to REG/i?n. Toward this 
goal, with a help of Proposition 16.31 it suffices to show that the distributional problem (A, a) does not belong 
to Aver-REG/n for a certain probability ensemble fi. We shall present a simple example of such language 
A, known as a REG/n-pseudorandom language [13] . Formally, a language L over an alphabet E is called 

REG / ' n-pseudorandom if, for every language A G REG/n over E, the function l(n) ~def — rgfq — \ 

negligible, where AAL denotes the symmetric difference between A and L (i.e., AAL = (A \ L) U (L \ A)) 



is 



Lemma 6.4 If A is REG /n-pseudorandom, then (A, u u 
is the uniform probability ensemble over S* (i.e., each a v 



i) is not in Aver-REG/n, where a un i — {fJ, un i.n}neN 
ii „ is the uniform probability distribution over Y7 l ). 



Proof. We prove the lemma by contrapositive. Let (A, n un i) be any distributional problem in Aver-REG/n 
over an alphabet E. Take a ldfa M, an error bound e 6 [0,1/2), and an advice function h satisfying that 
Probz^^ „[M([ h( x „ } ]) = A(x)} > 1 — e for every length n e N. For our convenience, we set e = h — e' with 



e' > 0. Now, let us define B = {x e E* 
then follows that 



M([ h(\x\) ]) = 1} an d consider the symmetric difference BAA. It 



Prob, 



, [x e BAA] = Probs 



Note that, since /i U ni,n is uniform, Prob x ^ un 
the value £(n) can be lower-bounded as 



£(n) = 



|(SA^)nE r 



\M{[ h U })jkA{x)]<- 



E BAA] equals \(BAA) n E n |/|E™|. From the above bound, 



Prob 



I~/"uni,ti I 



G BAA] - 



>e'. 
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This means that A cannot be REG/n-pseudorandom. □ 

With a use of Proposition 16 . 31 and Lemma HOI it becomes rather an easy task to prove Theorem 16. 11 since 
we already know from [T3] that the context-free language IP* is in fact REG/n-pseudorandom. 

Proof of Theorem 16.11 Assume that CFL C REG/ Rn. By Proposition [5111 for every context-free language 
A, the distributional problem (A, /x un ;) belongs to Aver-REG/n, where /x un ; is the uniform probability ensemble. 
Lemma 16.41 further implies that A cannot be REG/n-pseudorandom. In summery, no context-free language is 
REG/n-pseudorandom. This contradicts the fact that 7P* is a REG/n-pseudorandom context-free language 
P) . Therefore, it should hold that CFL £ REG/Rn. □ 



To close this section, we still need to prove Proposition 16.31 Its proof, in fact, follows immediately from a 
new characterization of REG / Rn given below. This characterization is a direct consequence of Yao's principle^ 
|14) and it is, to some extent, analogous to an existing result on one-way communication with public coins. 

Lemma 6.5 Let A be any language over an alphabet S. The following two statements are equivalent. 

1. A is in REG/Rn. 

2. There exist a Idfa M , an advice alphabet T , and an error bound e G [0, 1/2) that satisfy the following 
condition: for every probability ensemble {/U n }nGN over S*, there exists an advice function h : N — > T* 
such that Prob z: ^ A j 7l [M([ h( x n) ]) = .A(x)] > 1 — e for every length n G N. 

Proof. (1 => 2) Let A G REG/Rn over S, witnessed by M, e, T, and {P n }neN over V*; that is, for every 
length n G N and for every input x G S™, it holds that Probj,^^ [M ( [ £ ]) f£ A(x)] < e. Assuming that 
{MnjneN is a probability ensemble over £*, we claim that 

(*) for every length n G N, there exists an advice string y n G F n satisfying that 
Prob^„[M([ „• })^A(x)]<e. 

Using this special string y n , we can define the desired advice function h as h(n) = y n for each length n G N. 
This clearly yields (2). 

Let us prove Statement (*). Assume otherwise; namely, for a certain fixed n G N, Prob^^ [M ([ „ ]) ^ 
A(x)] > e for every string y G T n . Consider the product distribution v(x,y) = ^ n {x)D n {y) for any pair 
[x, y) G S* x r*. We shall estimate the value j n = Prob^ y )^i,[M ([ „ ]) ^ A(x)] in two different ways. This 
value 7„ is upper-bounded as 

In = ^2 fi n (x) ■ Proby^ Dn {M ({ I }) ^ A(x)} < J~] fj, n (x) -s = e. 

However, the same value 7„ is lower-bounded as 

In = ^£>„(y)-Prob x ^„[M([ l])^A{x)] > ^P„(y)-e = e. 

The above two bounds clearly lead to a contradiction. Therefore, Statement (*) must hold. 

(2 =>■ 1) Assume that there exists a ldfa M, an alphabet T, and a constant e such that, for every probability 
ensemble /i = {/x„} n6 N, there exists an advice function h H : N — > T* satisfying Prob x ^ Un [M([ h " n) ]) = 
-^O^)] ^ 1 — e - Fi x "£N arbitrarily. Let us consider the following two-player zero-sum game. 

Player 1 chooses y in T n and Player 2 chooses x in E™ randomly according to /i„. Player l's payoff 
P x . y is 1 if M([ I ]) = A(x), and otherwise. 

When Player 1 tries to maximize his payoff and Player 2 tries to minimize his own payoff, we obtain the 
inequality 

minmax} u n (x)P x y > minProb a; ^„ 7l [Af([ ,% } ]) = A(x)} > 1 — e. 



^Yao's principle is a randomized algorithmic interpretation of von Neumann's [9] celebrated minmax theorem in game theory. 
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By Yao's principle P3j, it follows that 

maxmin^ p n {y)Px. y = minmaxy^ p n {x)P x . y > 1 — e, 

n~ x *- — » n.„ 7/ * — » 



Pn K * ' Cii J/ 



where p n is a probability distribution over T n . We choose a particular p„ that satisfies the above equality, 
and we define D n (y) = p n (y) for each string y £ T n . By the choice of p n , for every string z £ E", we obtain 

Prob^ D „[M([ ; ]) = A(z)} > mmYjPn{x)Px, v > 1 - e. 

Therefore, with bounded-error probability, M recognizes A using the randomized advice {D„} n6 N- This means 
that A belongs to KEG/Rn. □ 



7 Brief Discussion 

Throughout this paper, we have shown strengths and weaknesses of deterministic and randomized advice when 
it is given particularly to weak models of one-tape linear-time Turing machines and finite automata. Such 
weak models have made it possible to prove collapses and separations among advised language families, as 
shown in Figure [TJ with no unproven assumption. 

Many class separations that have been proven so far are obtained in fact by discriminating two ma- 
chines' abilities to extract key information from a given piece of advice. However, there are still numerous 
open questions, which we need much more sophisticated arguments to solve. For instance, we would like to 
prove/disprove the following: 1-CJLIN/Un <£ KEG/Rn, CFL/n <£ l-C=LIN/7m, KEG/Rn £ 1-PLIN/Kn, 
and 1-PLIN /lin £ KEG/Rn (even more strongly, 1-PLIN /lin <jt CFL /Rn). 

In other research directions, one of the challenging tasks is to prove/disprove that CFL(fc)/_Rn £\ CFL(fc + 
l)/n for each index k E N+, where CFL(fc) = {f\ fe =1 h \ i £ [k},Li £ CFL} 13 . Moreover, a thorough 
investigation on Aver-REG/n is certainly another important challenge, which may expand an existing scope 
of formal language and automata theory. 
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